1. Field of the Invention
Embodiments of the invention described herein pertain to the field of computer systems. More particularly, but not by way of limitation, one or more embodiments of the invention enable a preemptive neural network database load balancer.
2. Description of the Related Art
Load balancers are software or hardware components that are used to spread tasks between multiple computing resources. Load balancing is performed to obtain scalability, decrease latency, and maximize performance for example. Load balancers may be utilized with server farms or clusters. Many load balancers can operate when a given server fails or during periods of server maintenance for example. Providing access to the computing resources when a server is not accessible allows for increased availability, or “up time” of the computing resources. Many types of load balancing are currently used including round-robin, least connections, least response time, least bandwidth, least packets, source Internet Protocol (IP), token and Uniform Resource Locator (URL) hashing for example.
Current scalable database clusters rely on load balancers that are reactive and not predictive. All of the algorithms mentioned in the previous paragraph as reactive algorithms. This results in poor system performance and/or increased hardware costs to account for the inefficiency of the load balancing algorithms currently in use. Load balancing algorithms currently in use do not preemptively assign incoming tasks to particular servers based on predicted Central Processing Unit (CPU) and/or predicted memory/disk/network utilization for the incoming tasks. In other words, the currently utilized algorithms are not preemptive. Furthermore, in architectures that include a heterogeneous mix of writeable and readable database servers, i.e., master and slave database servers respectively, there are no known load balancers that preemptively schedule tasks based on the read or write characteristic of a particular task. Specifically, there are no known load balancers that direct write-based requests or tasks to a master for example. Furthermore, there are no known load balancers that utilize a neural network to learn and predict which read-based tasks will utilize predicted amounts of resources such as CPU and/or memory/disk/network and assign the task to a database server in a cluster based on the predicted utilization.
The most basic algorithms for load balancing database clusters include reactive algorithms such as round robin or least connection. These load balancing algorithms consider all database servers in a cluster as equal and distribute client requests between the database servers in a round-robin manner or based on the information about the number of open connections. Round robin algorithms spread the incoming tasks to the next server in a cluster regardless of the predicted resource utilization of the incoming task. Connection based algorithms spread the incoming task to the server with the least connections regardless of the predicted resource utilization of the incoming task. Neither of these algorithms take into account the particular resources available to each server for example, the number of CPUs in a given server or the amount of memory to predict the future utilization of the servers. Likewise, these methods do not take into consideration the difficulty of tasks running on the servers and their influence on the resource utilization of the server. Current load balancing methodologies also do not take into account the current database characteristics such as the number of records, lookups, images, Portable Document Format (PDF) files, Binary Large Objects (BLOBs) and the widths of the fields for example and hence cannot possibly predict how long a particular task utilizing these parameters will take to execute or how much memory the task would consume. The other algorithms listed above likewise are reactive in nature and in no way predict how long a particular request or task will take, or how resource intensive the task will be in order to choose a server in a cluster to direct the task.
An example of poor distribution occurs when utilizing known load balancing algorithms when an incoming task obtains a connection to a database server executing a “resource-hungry” request (e.g. matching, complicated search, mass record deletion) rather than to a database server executing several simple requests (e.g. record retrievals). In this case traditional load balancing methods lead to asymmetric load balancing between database servers in a cluster. This results in resource allocation that is not optimized for the database application in use. Specifically, the results which occur from the spreading the tasks using known algorithms are random since there is no estimation on a per task level to even resource utilization between servers in cluster.
For example, given a two server cluster, if four incoming tasks include two resource intensive tasks and two light resource utilization tasks, then in the round robin case, there is a good chance that the two resource intensive tasks will execute on the same server in the cluster. In this scenario, the two light resource utilization tasks will execute quickly on the other server which will then stand idle while the other server runs completely utilized. Depending on the order that the tasks arrive, it is possible that each server will obtain a resource intensive task and a light resource utilization task. Thus, the result is random since the result depends on the order in which the tasks arrive at the load balancer. Likewise with the least connection algorithm, one can readily see that the server with the least connections may be executing at least one task that may for example be executing an extremely resource intensive task that may take a tremendous amount of CPU. Using this algorithm, an incoming resource intensive task is still directed to the server with the least connections. Hence, the results of this load balancing algorithm are random since the “size” of the tasks has nothing to do with the number of connections that a server in the cluster currently has.
In addition, as software implementing particular tasks changes over time as companies improve or otherwise alter the application software, there are no known load balancing algorithms that update to better predict resource utilization based on the new task performance characteristics. For example, if the software implementing a particular task becomes more efficient and then utilizes fewer resources for a particular operation, no known load balancing algorithm updates a predicted utilization parameter associated with the particular task. Alternatively, if a particular task is altered to add functionality, which tends to require more resources to operate, there is no known load balancing algorithm that updates a predicted utilization parameter associated with the particular task. Under either scenario, the software changes provide altered predictions for task execution that are not taken into account by any known load balancing algorithm.
Even if known systems were to utilize traditional linear methods of correlating input parameters with predicted resource utilization, these would still not provide satisfactory results since small variations of one input parameter may radically alter the required resource utilization for a particular task. As such, any non-learning based load balancing algorithm would be limited in the quality of predicted utilization and would in general be as haphazard as a round-robin scheme or least connections schemes.
The description of algorithms above is applicable to software based solutions or hardware based solutions that are available from numerous vendors. Although hardware based solutions may be dynamically updated with new firmware, essentially their operation relies on one of the algorithms previously discussed. Specifically, there are no known hardware solutions that preemptively load balance.
For at least the limitations described above there is a need for a preemptive neural network database load balancer.